SYSTEMS AND METHODS FOR SYNCHRONIZING 
MULTI-MODAL INTERACTIONS 

ABSTRACT OF THE DISCLOSURE 

System and methods for synchronizing interactions 
between mono-mode applications, different modes of a multi- 
modal application, and devices having different UI 
modalities. In one aspect, a multi-modal shell coordinates 
multiple mode processes (i.e. modalities) of the same 
application or multiple applications through API calls, 
whereby each mode process registers its active commands and 
the corresponding actions in each of the registered 
modalities. The multi-modal shell comprises a registry that 
is implemented with a command-to-action table. With the 
execution of a registered command, each of the corresponding 
actions are triggered to update each mode process 
accordingly, and possible update the registry to support new 
commands based on the change in state of the dialog or 
application. In another aspect, separate applications (with 
UI of different modalities) are coordinated via threads 
(e.g., applets) connected by socket connections (or virtual 
socket connections implemented differently) . Any command in 
one mode triggers the corresponding thread to communicate 
the action to the thread of the other application. This 
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second thread modifies accordingly the state of the second 
process mode. The threads are updated or replaced by new 
threads 
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